Vision Transformer models, such as ViT, Swin Transformer, and Transformer-in-Transformer, have recently gained significant traction in computer vision tasks due to their ability to capture the global relation between features which leads to superior performance. However, they are compute-heavy and difficult to deploy in resource-constrained edge devices. Existing hardware accelerators, including those for the closely-related BERT transformer models, do not target highly resource-constrained environments. In this paper, we address this gap and propose ViTA - a configurable hardware accelerator for inference of vision transformer models, targeting resource-constrained edge computing devices and avoiding repeated off-chip memory accesses. We employ a head-level pipeline and inter-layer MLP optimizations, and can support several commonly used vision transformer models with changes solely in our control logic. We achieve nearly 90% hardware utilization efficiency on most vision transformer models, report a power of 0.88W when synthesised with a clock of 150 MHz, and get reasonable frame rates - all of which makes ViTA suitable for edge applications.

How much computing power is needed at the edge? How much memory and storage are enough for AI at the edge? Minimum requirements are growing as AI opens the door to innovative applications that need more and faster processing, storage, and memory. How can today's memory and storage technologies meet the stringent requirements of these challenging new edge applications? Edge includes any distributed application where specific processing occurs away from the server, even if the data is eventually sent to a data center.

Edge is a term that refers to a location, far from the cloud or a big data center, where you have a computer device (edge device) capable of running (edge) applications. Edge computing is the act of running workloads on these edge devices. Machine learning at the edge (ML@Edge) is a concept that brings the capability of running ML models locally to edge devices. These ML models can then be invoked by the edge application. ML@Edge is important for many scenarios where raw data is collected from sources far from the cloud. Although ML@Edge can address many use cases, there are complex architectural challenges that need to be solved in order to have a secure, robust, and reliable design.

Today's digital world is an expanding frontier of emerging technologies. There are endless innovations, inspired by data, informed by data, enabled by data, and that create value from data. One thing we've seen more and more enterprises do to keep up with this digital revolution is the adoption of cloud services for a variety of IT functions, to an extent that modern approaches to building and running programs are often described as "cloud-native." According to Gartner, while only about 10 percent of enterprise-generated data is created and processed outside a traditional data center or cloud, this figure is expected to soar to 75 percent by 2025. The cloud alone simply isn't efficient enough to keep up with the volume and velocity of data that enterprises will be faced with as time goes on. So what is the missing piece to keeping up?

Model monitoring for predictive analytics at the edge begins with input, i.e. the data and how it is collected. I like to say, the Internet used to be a thing, but now, things are the Internet. In the Internet of Things, "things" are embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems. The data that is collected at the edge often needs to be processed in real-time in order to fuel predictive modeling or to reveal novel patterns in the data that may inspire questions we didn't think to ask about the things that we are monitoring. Some examples of edge applications are technologies like drones or self-driving cars, which operate autonomously through software controlled plans and onboard edge sensors, including GPS.

Nota, which provides technology to optimize AI models, announced that it has closed a $14.7 million Series B funding round. The company's technology is another important piece of the puzzle when it comes to helping resource-constrained edge devices run applications such as biometric identification. Participants in the funding round included Stonebridge Ventures, LB Investment, DS Asset, Intervest, and Company K Partners. The fresh funding comes roughly a year after Nota closed its Series A round with $6.7 million. Nota has raised a total of $23 million to date.

The job of an IT architect or administrator becomes more complex with each passing day. It hasn't been that long since an IT staff only had to deal with the resources found inside the walls of the enterprise. IT organizations must address the needs of a remote workforce, multi-cloud integration, and the rapid rise of edge computing and artificial intelligence. Edge computing places compute and storage capabilities into the real world, allowing an enterprise to generate insights and deliver value where it is most needed. Edge computing extends the reach of enterprise IT right to the point that data is generated, enabling new and impactful use cases that change the way that many businesses operate.

The saturation of mobile devices and ubiquitous connectivity has steeped the world in an array of wireless connectivity, from the growing terrestrial and non-terrestrial cellular infrastructure and supporting fiber and wireless backhaul networks to the massive IoT ecosystem with newly developed protocols and SoCs to support the billions of sensor nodes intended to send data to the cloud. By 2025, the global datasphere is expected to approach 175 zettabytes per year. What's more, the number of connected devices is anticipated to reach 50 billion by 2030. However, the traditional distributed sensing scheme with the cloud-based centralized processing of data has severe limitations in security, power management, and latency -- the end-to-end (E2E) latencies for ultra-reliable low-latency communications found in 5G standards are on the order of tens of milliseconds. This has led to a demand to drive data processing to the edge, disaggregating computational (and storage) resources to reduce the massive overhead that comes with involving the entire signal chain in uplink and downlink transmissions. New advances in machine learning (ML) and deep neural networks (DNNs) with artificial intelligence promise to provide this insight at the edge, but these solutions come with a huge computational burden that cannot be satisfied with conventional software and embedded processor approaches.

This article looks at the unique challenges introduced by Edge computing for AI/ML workloads, which can have a negative impact on results. It applies available machine learning models to real-world Edge datasets, to show how these challenges can be overcome, while preserving accuracy in the dynamic nature of Edge environments. The field of machine learning has experienced an explosion of innovation over the past 10 years. Although its roots date back more than 70 years when Alan Turing devised the Turing Test, it has not matured significantly until recently. Two primary contributing factors are the exponential growth in both compute power and data that can be used for training. There is now enough data and compute power (some in specialized hardware like GPUs/FPGAs) that new, real-world problems are being solved every day with machine learning.

If data starts at the Edge, why can't we do as much as possible right there from an AI point of view? The explosive growth in Edge devices and applications requires new thinking as to where and how data is analyzed, and insights are derived. New Edge computing options, coupled with more demanding speed-to-insight requirements in many use cases, are driving up the use of artificial intelligence (AI) and machine learning (ML) in Edge applications. Where AI and ML are applied (at the Edge or in a data center or cloud facility) is a complex matter. To get some insights into current strategies and best practices, we recently sat down with Said Tabet, Chief Architect, AI/ML & Edge; and Calvin Smith, CTO, Emerging Technology Solutions; both in the Office of the Global CTO at Dell Technologies. We discussed the growing need for AI and ML to bring sense to the large amount of Edge data that is generated today, the compute requirements for AI/ML in Edge applications, and whether such computations should be done at the Edge or in a data center or cloud facility. RTInsights: What are today's emerging trends, and how do AI and ML fit into the Edge discussion?